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{7,"<B>,""</B>;" , <I>; , "</I>/"<TABLE>"} 

procedure pcwrapHLRT (page P) 
scan to the line 7 in P 

while the next occurrence of // in P occurs before the next occurrence of t 
scan in P to next occurrence of /,; save position as start of item attribute 
scan in P to next occurrence of r h save position as end of item attribute 
scan in P to next occurrence of / p ; save position as start of price attribute 
scan in P to next occurrence of r p ; save position as end of price attribute 
return extracted , (item, price),. . . . .} pairs 



FIGURE 1 
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Element Name 


Element Description 


Vendor Name 


Name of the online shop 


Vendor URL 


The URL of the online shop 


Form URL 


The URL to submit search data 


Learning Domain 


The domain used to learn the vendor description 


Head 


The end of header position of vendor's product pages. 

This element to learnt to rpdupp thp nroHurt nanp 

searching space and thus shorten the product 
information extraction time 


Tail 


The start of footer position of vendor's product pages. 
Same as Head element, with this element the product 
information extraction time for Shopper Agent is 
shorten. 


Left Delimiter of Item 


The Shopper Agent uses this two delimiters to identify 
a product 


Right Delimiter of Item 


Left Delimiter of Price 


The Shopper Agent uses this two delimiters to locate 
a product's price information 


Right Delimiter of Price 
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Element 
Name 


Element Description 


Vendor Name 


800.com 


Vendor URL 


http://www.800.com 


Form URL 


http://www.800.com/search/srch^ 


Learning 
Domain 


Md 


Head 


1230 


Tail 


</BODY> 


Left Delimiter 
of Item 


/td width="12"> 


Right 
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Item 


</fon 


Left Delimiter 
of Price 


Your Price: $ 
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Delimiter of 
Price 


</b> 



FIGURE 5 



new U.S. Appln. based on Appln. Nos. 60/236,574 
& 60/299,360 for Online Intelligent Information... 
Express Mail Label No. EL864389877US 
Attorney Docket No.: 2102680-990101 
Gray Cary et al. - GTS/415-836-2500 
Applicants: Victor Hsieh Sheet 5 of 36 



140 





START 




1 


r 



210 



LABEL [POSITION VALUES] 
TRAINING PAGES 



220 



GENERATE SET POSSIBLE VENDOR 
DESCRIPTION CANDIDATES 



V 




240 



VALIDATE VENDOR 
DESCRIPTION CANDIDATE 




FIGURE 6 



new U.S. Appln. based on Appln. Nos. 60/236,574 
& 60/299,360 for Online Intelligent Information. . . 
Express Mail Label No. EL864389877US 
Attorney Docket No.: 2102680-990101 
Gray Cary et al. - GTS/415-836-2500 
Applicants: Victor Hsieh Sheet 6 of 36 
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<HTML> 




< 1 1 1 ll><d>a oimpie r roaucc uataiogs</D></ 1 1 1 i_t> 


Model 
Number 


PRICE 
(US$) 


S DUU I *^ 

<H2> MD Price </H2> 
<TABLE BORDER=1> 

<TR BGCOLOR=ORANGE><TH> Model Number </TH> 
<TH> PRICEfUS$K/TH></TR> 

^ l 1 1 1 1 \ 1 ^JL_l \J\J*p I J III i 1 1 \ 

<TR><TD><B> HM381MD</B></TD><TD><I>399 95</I></TD></TR> 

^ 1 1 \ 1 LJ ^ LJ ^ I I I V Iv VJ 1 1 V 1 LJ ^/ U"^ I I—' ^ 1 LJ 1 WW \J • w *J / 1 I 1 LJ / 1 1 x ^ 


HM381MD 


399.95 


MD2070 


599.95 


MD203 


249.95 


MDR3 


399.95 


<TR><TD><B> MD2070</Bx/TD><TD><I>599 95<I></TD></TR> 

^ 1 I \**^ ^ I LJ LJ ^ 1 V 1 LJ t_ W t \J 1 LJ t 1 LJ 1 LJ 1 \J V ^/ i \J \J 7 1 / 1 LJ /II \ 




<TR><TD><B> MD203</B></TD><TD><I>249.95</I></TD></TR> 






<TRxTDxB> MDR^</B></TD><TD><I>399 95</l></TD></TR> 

^* 1 i ^ 1 LJ ^ ^ LJ *^ l V 1 LJ 1 \vJ ^/ LJ ^/ 1 l_J^ 1 LJ ^ J vJ »J %J • v-/ vj ^/ 1 ^/ 1 LJ ^/ 1 1 






</TABLE> 

fit 
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<HR WIDTH=200 ALIGN=LEFT> 






<P> 






<B> End Of The Product Catalog </B> 
</BODY> 






</HTML> 
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procedure exec H i_RT (wrapper (h, li, n, l Pl r p , t), page P) 
m «-0 

scan in P to line h 

while the next occurrence of /, in P occurs before the next occurrence of t 
m m + 1 

scan in P to the next occurrence of /, in P; save position as b m ,j 
scan in P the next occurrence of n ; save position as e mji 
scan in P to the next occurrence of /„ in P; save position as b m , p 
scan in P to next occurrence of r p ; save position as e m>p 
return label {. . ., «b mi , e mi >,<b mp , e mD »| 

FIGURE 25 



Procedure learnHLRT (examples e) 

1. Generate the candidate sets Cands r (l, e), Candsi(l, e ), Cands r (p, e), Candsi(p, 
e). 

2. Enumerate the cross product of these candidate sets; each element W = (h, li, n, 
l p , r p , t), of this cross product is a wrapper. Halt if W is satisfactory i.e., execLR(W, 
P n ) = Ln, for every (P n , Ln) e s. 



FIGURE 26 
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/* 

* The main problem 
*/ 

procedure learnHLRT (examples e) for each 1 <k <K 
for each u e cands,(p, s): if valid,(u, p, s) then l p ^- u and terminate this loop 
for each u e cands r (i, s): if valid r (u, i, e) then n<-and terminate this loop 
for each u e cands r (p, e): if valid r (u, p, s) then r p <-u and terminate this loop 
for each u n e candS|(i, s) 
for each u h e cands h (s) 
for each u t e cands t (s) 
ifvalid lhit ( u nj u h , u t , e) then 

li<-U| h h<-u h , K-u t , and terminate these three loops 
return HLRT wrapper (h, lj, r if l p , r p , t) 

/* 

*Generate a set of candidates for left delimiter of price attribute 
*/ 

procedure candS|(attribute price, examples s) 
return the set of all suffixes of the shortest string in neighborS|(price, e) 

r 

*Generate a set of candidates for right delimiter of attribute a. Here a could be item or prk 
*/ 

procedure cands r (attribute a, examples s) 

return the set of all prefixes of the shortest string in neighbors^, e) 

/* 

* Generate a set of candidates for a page's head 
*/ 

procedure cands h (examples s) 

return the set of all substrings of the shortest string in heads(s) 

/* 

*Generate a set of candidates for a page's tail 
7 

procedure cands t (example s) 

return the set of all substrings of the shortest string in tails(e) 

/* 

*Determine whether a particular candidate for the left delimiter of price is valid 

This procedure applies constraints C3 

7 

procedure validj(candidate u, attribute price, examples e) 
for each se neighbors^price, s): if u is not a proper suffix of s then return FALSE 

Figure 27 A 
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for each s e tails(s): If u is a sub string of s then return FALSE 
return TRUE 

/* 

'Determine whether a particular candidate for the right delimiter of item or price is 
*valid 

*This procedure applies constraints C1 and C2 

*/ 

procedure valid r (candidate u, attribute a, examples s) 
for each s e attribs(a, s): if u is a sub string of s then return FALSE 
for each s e neighbors(a, e): if u is not a prefix of s then return FALSE 

return TRUE 

/* 

'Determine whether a particular combination of candidate uh, ut, and uli for h, t, and 
*li respectively are satisfactory. 

7 

procedure valid| iihtt (candidates tin, u h , u t , example e) 
for each s e heads(e) 

if is not a sub string of s then return FALSE 
if U M is not a proper suffix of scan (s, Uh)then return FALSE 
if Ut occurs before U^ in scan (s, Uh), then return FALSE 
for each s e tails (e) 
if Ut 1 S not a sub string of s then return FALSE 
if Uli occurs before Ut in s then return FALSE 
for each s e seps(s) 

if Uli is not a proper suffix of s then return FALSE 
if Ut occurs before Uli in s then return FALSE 
return TRUE 

/* 

'Return a set of containing all values of a attribute, either item(i) or price(p) in each 
*example 
7. 

procedure attribs(attribute a, examples e) 

return U (Pn , L n) e e(Pn[b m . a , e m , a ] l<. . (b m , k , e m , k ), . . . > e Ln} 

/* 

*Return all strings to the left of an attribute, whether these strings are in the heads 
* or the bodies of the pages 

7 

procedure neighborS|(attribute a, examples s) 

if a=i then return seps (i, s) U heads (e) else return seps (a, s) 



FIGURE 27B 
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START 



FIGURE 30 



Call SQLAccEnv 
to allocate environment workspace 
(provides HENV handle) 



Call SQLAIIocConnect 
to allocate connection workspace 
(provides HDBC handle) 



Call SQLConnect 
to connect to the database 



Call SQLAIIocStmt 
to allocate a statement workspace 
(provides HSTMT handle) 



Call ODBC member functions 
(SQLExecDirect, SQLFetch, SQLGoData) 
to perform a database operation 



Call SQLFreeStmt 
to deallocate statement workspace 
(invalidates HSTMT handle) 



Call SQLDisconnect 
to disconnect to the database 



Call SQLFreeConnect 
to deallocate connection workspace 
(invalidates HDBC handle) 



Call SQLFreeEnv 
to deallocate environment workspace 
(invalidates HENV handle) 



STOP 
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Call member function 
s.GetHttpConnection 
(creates CHttpConnection object c) 



FIGURE 31 
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Call member function 
c.OpenRequest 
(creates CHttpFile objectf) 
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Call member function 
f.SendRequest 
(sends the POST request and formdata) 
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Call member function 
f.Read 

(returns chunk of response) 
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